1. Field of Invention
The current invention relates generally to apparatus, systems and methods for processing image data. More particularly, the apparatus, systems and methods relate to processing stereo image data. Specifically, the apparatus, systems and methods provide for processing stereo image data (including ad hoc stereo) to detect changes without generating ortho-image data, or constraining image collection to a precisely repeated geometry.
2. Description of Related Art
In a good stereo pair, humans readily fuse the two images and perceive a 3-D scene. The relief may be exaggerated, but our brains are comfortable with the presentation. Similarly, stereo correlation algorithms used for automatic terrain extraction operate nicely on good stereo pairs. But localized differences between stereo image pairs can cause headaches for humans and correlation software alike.
For human observers, these differences confound our natural stereoscopic vision. It is immediately apparent that something isn't right. The differences also confuse image-matching terrain extraction software. The results include spikes, wells and other elevation anomalies that previously required manual editing to correct.
In performing change detection one generally has two choices to eliminate apparent image differences due to collection geometry: 1) constrain the collection to precisely replicate the geometry on subsequent passes, or 2) perform ortho-rectification on images taken at differing geometries. Constraining geometry may not be practical, and our method makes it unnecessary. Ortho-rectification adds time-consuming and expensive steps to the process and deliberately introduces pixel distortions (spatial and spectral) that can produce erroneous change results. We dispense with these steps.
Sometimes ortho-rectification is performed without compensation for topology. Without using a DEM (digital elevation mode, e.g., bare Earth model) in the ortho-rectification process, differential layover distortions persist that will defeat change detection, except in the case of very large pixel sizes (like LANDSAT, MODIS, AVHRR), or very flat terrain. To remove these distortions requires either as many control points as there are pixels (completely untenable!), or true ortho-rectification with a very accurate DEM (which may not exist at suitable resolution).
Any process to re-project an image, whether it compensates for relief or not, is a deliberate distortion of image pixels that fundamentally changes the spatial and spectral character of the image. In change detection, this could result in erroneous results. Epipolar rectification is a fairly minor rotational re-sampling of pixels in the image plane, and does not introduce the large spatial distortions and spectral blending of ortho-rectification.
Ortho-rectification modes include parametric methods and non-parametric methods. The methods applied vary in the quality of resultant ortho-image product. The non-parametric methods generally use 2-dimensional polynomial transformation functions. One of the simplest ways to ortho-rectify an image (and least accurate) is to apply a polynomial function to the surface and adapt the polynomials to a number of ground control points (GCPs). This procedure can only remove the effect of tilt, and can be applied on both satellite images and aerial photographs.
A variety of methods can be used to generate different ortho-rectification models. For example a projective rectification ortho-rectification process can be used to geometrically transform between the image plane and a projective plane. A differential ortho-rectification process can be used to assign gray values from the image (usually an aerial image) to each cell within an ortho-photo. Sensor models are typically classified into two categories: physical and generalized models. The relationship between image and corresponding ground coordinates is established through a physical imaging condition model in the form of collinearity conditions.
Rational function model (RFM) rectification is yet another way to generate ortho-image data. The RFM sensor model describes the geometric relationship between the object space and image space. It relates object point coordinates (X, Y, Z) to image pixel coordinates (r, c) or vice versa using rational polynomial coefficients (RPCs).
Orthorectification algorithms are often performed in conjunction with a re-projection procedure, where rays from the image are re-projected onto a model of the terrain. Fundamentally re-projection can be done in two ways: forward projection (direct projection) and backward projection (indirect method). In the first case of forward projection, the pixels from the original image are projected on top of the DEM of the 3D model and the pixels' object space coordinates are calculated. Then, the object space points are projected into the ortho-image
In the case of backward projection, the object space X, Y coordinates related to every pixel of the final ortho-image are determined. The height Z at a specific X, Y point is calculated from the digital elevation model (DEM) or the 3D model and then the X, Y, Z object space coordinates are projected in the original image in order to acquire the gray level value for the ortho-image pixel.
Often, such ortho-rendering processes require many mathematical computations and are very time-consuming. In many cases, those processes require iterative processing that measurably degrades image quality and resolution—especially in cases where thousands of sub-images are being rendered. In cases where the imaging data can be processed automatically, that data is often repetitively transformed and sampled—reducing color fidelity and image sharpness with each successive manipulation. If automated correction or balancing systems are employed, such systems may be susceptible to image anomalies (e.g., unusually bright or dark objects)—leading to over or under-corrections and unreliable interpretations of image data. Therefore, there exists a need for better change detection through stereo image matching.